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(NTRODUCnON 

Arithmetic &nd logical procesaors in com- 
put«ra of the *'secODd generation" and earlier 
tended to be unBophisticated insofar aa their 
Ughly aerial nature of instnicUon execu- 
tion waa concerned. Furthennore, the bottle- 
neck created by a relatively alow core 
memory with a single-access port made the 
problem of enhancing the processor'a speed 
uninteresting. With the advent of such tech- 
niques as multiple-port interleaved memo- 
ries, semiconductor memories, and the use of 
more fast, local registers (either program- 
mable or cache), we have the capability of 
tranamitting operands and mulU between 
processors and memory at a much faster 
rate. The ability to provide a corresponding 
rate of instruction execution, then, depends 
on the speed of the processor. 

Techniques for enhancing the speed of a 
processor by "look-ahead" are examined in 
thia paper. The term look-ahead derives 
from a class of schemes in which programs 
for the processor are specified in a conven- 



tional, serial manner; however, the processor 
can look ahead during execution and exe- 
cute instructions out of sequence, provided 
no logical inconsistencies arise as a result of 
doing 80. The advantage of look-ahead is 
i^that several instructions can be executed 
concurrently, assuming the processor has 
sufficient capabilities. Deugns of specific 
look-ahead processors have been presented 
in (AST, Th. To). 

Using the diagrams in Figures 1 and 2, 
this paper will model a computer with a 
look-ahead processor. We are concerned 
mainly with the processor here, and not the 
overall system. Note that the processor 
contAtna a number of focoi reguttert for the 
storage of data, a number of fundion uniu 
for operating on this data, and an instruc- 
tion buffer, or window, for storage of instruc- 
tions. We use the term "window" because 
the instruction buffer can be viewed as 
looking onto a small segment of the prognun 
in execution. 

We assume that the programs to be exe- 
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cuted specify aeriai exeaUion. That ia, the 
coirect semantics • of execution are defined 
by the execution of one instruction at a time, 
in the order specified. It is the task of the 
)ook>ahead processor to determine which in- 
structions can be executed concurrently 
without changing the semantics. 

Once a processor has determined which 
instructions can be executed concurrently, 
it must assign them to the available physical 
function units. Hence, the processor performs 
two main tasks: 

1) deie^ion oj poraii«lum— determining 
which instructions may be executed 
concurrently, and 

2) aeheduling — assigning concurrently 
executable instructions to funcUon 
units. 

We shall see that the detection of parallel- 
ism is largely a machine-independent task, 
whereas scheduling — ^at least optimal sched- 
uUng— must generally take into account the 
specific number of function units available. 

Some additional tasks which might also 
be performed by such a processor are: 

3) register assignment and renaming, 
and 



4) modifying code according to algebraic 
identities. 

We shall not be concerned with these tasks 
initially, but will comment on them later. 

The emphasis here will be on schemes that 
approach optimallty in the detection of 
parallelism. Many definitions of optimality 
are possible, depending on the choice made 
among a set of possible "ground ntles." 
Several possibilities will be mentioned here, 
and we shall give techniques for approach- 
ing optimal look-ahead for some of these. 

In discussing optimality, there are two 
major categories: 

1) Global optimality— optimality with 
respect to the execution of entire 
programs', and 

2) Local optimality— optimality with 
respect to the contents of the window 
only. 

Discussing global optimality for general 
programs appears to be an extremely diffi- 
cult problem. Indeed, it is even difficult to 
say what we mean by "optimal" in this case, 
since the class of all possible programs is 
extremely large. In contrast, by suitably 
restricting our model we can say some 
things about local optimality. 

A further division occurs within the con» 
text of local optimality. There appear to be 
two types: italic and dynamic. This dis- 
tinction arises from the fact that the window 
contents may be constantly changing. 
That is, when one instruction in the window 
has been completely executed, it may be 
"retired" and a new instruction brought in. 
Then what started out as an optimal 
strategy before the new instruction was 
added may end up being nonoptimal. This is 
what we mean by ".dyn*«n>c." 

As with global optimality. local optimality 
in the dynamic ease is difficult to define. 
We therefore restrict ourselves to the static 
case at present. Thus, we arc inUrested in 
optimal behavior with respect to fixed 
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window oontenU. aa an approximation to a 
true local optimuni. The approximation will 
be best in caaea where the rate of change of 
the window is alow in compariaon to the 
execution rate. This occura, for example, if a 
program loop is entirely contained within 
the window and the loop executea a aub> 
atantial number of iterations. 

We ah&ll now specify the model in more 
detail. Aaaume the window contents repre> 
aent a aegmeot of the program being exe- 
cuted, with elements of that segment being 
statements of the fom (here A is to be read 
asasubBcript): 

or a; if GkU) then goto a' 
or a: goto a' 
or a: exit 

Here a and a' represent instruction labels, 
which are implicitly associated with the 
instructions; t, i, and k represent registers 
local to the processor; Fk represents one of a 
aet of funcUona (such as "add," "multiply," 
ete. We assume that eaeh function has two 
arguments for simplicity. It will become 
apparent that thia causes little loaa of 
generality.); represents one of a set of 
teat predicates (auch as "compare to lero"); 
and "exit" signiiies to the control that the 
next statement is not in the window (not 
the end of the program). Hence, "exit" indi- 



cates to the control that further instructions 
must be fetched. 

We will not concern ourselves with the 
mechanies of fetching instructions or oper- 
ands from the central memory, but will 
assume that thia is handled by a mechanism 
external to that of the look-ahead. It 
Bufficee to say that this fetching is heavily 
overlapped with instruction execution. Mul- 
tiple-memory modules and interleaving may 
be used to achieve a sufficiently high rate of 
instruction flow. The fetching of an operand 
for register t can be represented as an assign- 
ment I f fc , where Fk takes no arguments, 
and the storage of an operand can be repre- 
sented by assigning one or more local regis- 
ters to play the role of a storage buffer. 

THEOREnCAL BASIS 

We present here a theoretical basis for the 
construction of look-ahead schemes, as well 
as certain asaumptiona that are oiade for 
convenience in comparing various scbemtt 
with respect to optimality. 

We will call a statement with specific in- 
dices A, t, i, k an operaHon, Each statement, 
then, is called an instance of an operation. 
Operations corresponding to the "if" in- 
struction discussed in the introductory 
section will be called deeiaiona. In this sec- 
tion, we will be primarily concerned with the 
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decinon-free caae, in which decisions \vill not 
play a role in look-ahead. 

It ifi aaaumed that the reader under* 
Btanda what is meant by oequenlial execution 
of a program or a program segment. Wc 
therefore present the following asaumpUona 
without further explanation. 

Attumption 1: We are interested in pos- 
sibly.parallel executions of segments that 
are equivalmt to sequential execution, in 
the sense that the sequences of register 
contents are the same as they would be in 
the sequential caae. 

One consequence of this assumption is 
that if the issuance of instructions were sud- 
denly stopped, for example by a program in- 
terrupt, then the resulting state of the ma- 
chine would be invariant after all statements 
in the segment were completely executed. 

The following assumptions (2 and 3) need 
not hold. Their purpose is to establish 
ground rules for comparing various schemes, 
Asrumptian 2: We assume that no non- 
trivial relations (such as function equality) 
hold between two functions or tests of 
different indices. 

This assumption is conservative, and it 
aimpiy means certain algebraic reductions 
that might otherwise preserve equivalence 
are not allowed. 

Autumptum 3; We assume there are no in- 
essential instructions; that is, no two in- 
structions compute or test identical values. 

Like its predecessor, this assumption is 
conservative. It is the informal equivalent 
of the "repetition -free" assumption in 
{KM2, Ke]. Its purpose is to prevent con- 
sideration of the assignment of infeasible 
program-optimization tasks to the processor. 
We may assume that preprocessing has 
already removed any inessential instruc- 
tions. 

Suppose b b an operation. We define the 
domain reguLera D{b) and range reffiaters R{b) 
as follows: If b is an instance of 

» *- k) 

then D{b) - \j, k\ and Rib) - If b is 
an instance of 

if GkU) then goto a' 

then Z)(b) - \j\ and R{b) - 0. If b and c 



Are two different operations, then we write 
amflici (b, c) if, and only if, either 

m n Die) ^ 0 
or R{c) n X)(b) ^ 0 
or ft(b)n/2(c) p4 0. 

Otherwise, we write no-conjUct{b^ c). 

It is a fact that ^)«£t no-eonfiict (b, e) is a 
aufficienl condition for a pair of operations 
b, c to be executed concurrently, or in either 
order, asynchronously (i.c, without regard 
to timing), and still remain equivalent to 
sequential execution in the sense described in 
Assumption 1. This is intuitively clear, but 
has been observed in |B], and given formal 
treatment in |Ke, KM2|. 

To see why this condition is generally 
ntuttary, if b and e are executed concurrently 
and 72(6) fl D{c) 5^ 0, then some value 
that c fetches is generally dependent on 
whether b has stored anything into a register 
common to /2(b) and D{c), Similarlv, if 
ft(b) n R{c) pi 0, then the net result is de- 
pendent on whether b or e was the last to 
store something into a register common to 
both R{b) and R{c). The condition would 
not be necessary if the value stored by b 
happens to be the same as that fetched or 
stored (respectively) by c. We can see, 
however, that either case would constitute a 
violation of Assumption 3. Hence, we will 
assume that this condition is both necessary 
and sufficient for the concurrent execution 
of two operations. 

In discussing techniques for detecting 
parallelism, we are therefore interested in 
schemes that preserve the order of opera- 
tions b, c in which eanfiici (b, c) is the condi- 
tion. We will also assume for now that b 
and c arc not decisions, for to do otherwise 
would require some method of specifying 
precisely what it means to interchange 
them. 

Combined with the serial ordering of 
instructions, the conflict relation gives rise to 
a preee(ience rdaiion. We say preecda {at , oj) 
if, in any execution, a< must be cxecutcid 
before a^. The precedence relation may 
be deternuned from both the conflict rela- 
tion and the given serial ordering as 
follows: assume that «i , St . * " . is the 
serial ordering specified by the program. 
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FiooAC 3. Tha F relation for Exani|iio I and Ita 
graph. (The conflict ffrapb la obtained by mak- 
ing each arrofr bidireotional.) 




Fiouac 4. The "preeadea" ralation for EiampIe 
I ood ita graph. 

Let , fly) be true if, and only if, i < j 
BDd amfiid, («<, Then preudet is the 
"tranaitive doaure" of P; that ia, pruedu 
Ui . «>) if. and only if, there is a sequence 
» » «i < H < • • • < ir - ; such that 
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covero" relation for Example I 



eonjlict (a,, , aj, an^id , a.,), . . • , 

confiicl , a..). 
Reasonably efficient algorithms for . com- 
puting the tranaitive closure of a relation 
are known; cf. [War|. 

Actually, another relation is more im- 
portant than preeedcM for our purposes. This 
relation, which we call coiwra, is the "cover" 
of the relation preeedea. (It is the cover of P 
as well.) That Is, covcra ia the smallest 
relation whose transitive closure Is precetUs, 
This condition is obtained by simply remov- 
ing redundant arcs which are implied by 
tranativity. An algorithm for computing 
"covers" is diBcuBsed in (AGU). wherein the 
"covers" relation is referred to as the "tran> 
sitive reducUon." Studying the example in 
Figures 3, 4, and 5 should make the meaning 
of these relations and their computation 
reasonably clear. For a formal analysis, 
aee [Ke|. 

As a further basis for comparing different 
look-ahead schemes, we shall assume that 
instructions are examined ocquenticUy in the 
generation of operations. In other words, 
we hypotheaixe a unit, called an operation 
ittuing unit (OIU), which examines the 
contents of the window one instruction at a 
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time and which either issues the correspond* 
ing operation to a function unit, or decides 
to defer the tssuance for some reason. For 
now, we assume that an operation is iasued 
by transferring the indieet of the registen 
involved to the appropriate function unit, 
which then operates on the values in the 
registers indexed. 

The issuance of operations proceeds con- 
currently with the entrance and exit of 
instructions in the window. It is possible to 
examine instructions in parallel; however, 
we contend that sufficient speed can be 
achieved by sequential examination. Fur- 
thermore, parallel examination of instnic- 
tions appears to increase, unduly, the com- 
plexity of the control. 

If the issuance of an operation correspond- 
ing to the instruction being examined is 
deferred, we say that the operation is pend- 
ing. If the operation has been successfully 
issued, wc say that it is being executed until 
the time it is eompleted. 

To summarize this section, we may state 
the following: 

^ Principle of Optimaliiy for operation issue 
(decision-free case): Whenever c is an 
operation correspohding to an ijiatruction 
in the i^'indow, and there ia no operation 
b which is either being executed or is 
pending execution such that eonJiiel{b, c), 
then operation c shoujd be issued. 

BfMENTARY SCHEMES 

We shall now examine some schemes for 
detecting parallelism by using the principle 
of optimality stated in the previous section. 
For initial simplicity, we assume that the 
window does not contain any decisions; 
rather, detection of a decision is accom* 
plished external to the window, and it 
causes the transfer of instructions into the 
window to halt until the decision is resolved. 

The fiiBt scheme will be called the oimpU 
tndieaior echenu. As we shall see, this method 
ia closely related to a scheme discussed in 
|Tol, but we have elaborated it slightly for 
the sake of explanation. With each register t 
is associated an indicator regiaUr, Wt , which 
holds an encoded value from the set 1 1, 0, 
— 1,-2, , —JV I where iV is the maximum 



number of concurrently-executable opera- 
tions. If ITi « 1, then an operation b Is In 
progress such that i 6 R{b). If o -m, 
then there are m operations 6 io progress 
with I 6 Dib), If an operation b is in progreaa 
with t € D(b) and t € R{b), then it will be 
the case that fTi « 1 by convention. In addi- 
tion, a one-lMt register Bk is associated with 
each function unit f a . « 1 if an opera- 
tion using function unit Fa ia in progreas, 
and Ba » 0 otherwise. 

The operation-issuing unit (OIU) ex- 
amines the instructions in the window ae* 
quentially, and if the necessary conditions 
are satisBed, it issues the instruction to the 
appropriate function unit. If the instruction 
is 

then the conditions which must be satisfied 
arc 

IT, « 0 

< 0 
- 0 

Once the operation is iasued, and before the 
next instruction is examined, the OIU sets 
- - I, and seU W,*-Wf- 1, and 
H'* H^* — I (unless ; or Jk = i, in which 
ease >V< is set to 1 as discussed above). 
When the operation b completely executed, 
Bk and Wi are set to 0 and K'/ «- IT/ + 1 
and Wi^Wk-¥l (unless ; or fc «» i, as in the 
previous case). 

The completion of the operation is de- 
termined in the synchronous case by a spe- 
cific elapsed time, or, in the asynchronous 
case, by notification by the function unit 
itself. At this point in the discussion this 
detail is unimportant. 

Example I: We consider the following win- 
dow contents as part of a running example: 

tn^^rucfton operofton 
1 - F.(2, 3) (a) 



5 ^ f,(l. 2) 
4 - F,(2, 2) 

3- f,(l,4) 

6 ^ F,(5, 6) 
1 Fi(2, 3) 

4- r«(2, 5) 
3-F,(1.4) 
6 F,(5, 6) 



(b) 
(c) 
(d) 
(e) 
W 
(S) 
(d) 
(0 
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PiaDaK 0. UmioB of Example 1 ueins aitnple in>* 
dieaior ocbAmo and one Ft unit, aasuming unit 
flxeeution Umao for all operations. 

The operation names shown here will not 
play a role in this discussion until we treat 
the problem of optimal schemes without 
associative search, in the section so titled. 

In Figure 6, we illustrate a possible timing 
diagram for the simple indicator scheme 
with one Fi unit, which each function is as* 
sumed to require one unit of time. We em> 
phasize that this assumption ia made for the 
purpose of illustration only. We also assume 
that the time required for actual scanning ia 
negligible. A description of the scan in this 
caae is as follows: At ( « 0, fli starts. Since 
ot depends on «i , «t is not issued and the 
scan stops. At 1 « 1, ot completes and »t 
starts. The scan then continues and it 
starts. As 04 depends on t» , the scan stops. 
At < » 2, S4 staxta, but «i requires the same 
function unit as <4 » so si stops the scan. At 
I » 3, at starts, etc. The time required to 
completely execute the window contenu is 6. 

In Figure 7 we illustrate the timing dia- 
gram for the simple indicator scheme using 
two Fi unita. The scan in this case is similar 



I I I I i I 
t-« 1 a s 4 - s 

Fiooac 7. Timins uatns oimple iDdieator oeheiao 
and itro ^1 units. 

constraint involves the introduction of viV- 
tual Junction uniUt. (A simikr concept ia that 
of rucrvo^um ttaUona [To].) ^ 

A virtual function unit is used to repre- 
sent an operation which could be in progress, 
but which might not be due to the unavail- 
ability of a real function unit We may think 
of such operations as forming a queue, which 
is served by the real function unit when it 
becomes available. This also seems to be a 
eonveiuent way of organizing the allocation 
of several function units of the same type. 

The advantage of the virtual function 
unit method is that it does not allow the 
issuance of operations to be impeded by 
busy function unita, but only by register 
conflicts. Of course in practice, even the 
number of virtual functions will be bounded 
by' hardware considerations, and a counter 
that indicates the number of unita available 
would be used, halting the issuance of opera- 
tions when-'the couiit becomes aero. 
Figure 8 illustrates the same example, 



, . t x-ijjurc e iiiuobickva w>~.»f— 

to the previous one, except that at « - X except that there is one Ft unit and two vir- 



both ii4 and ai can start. This has the effect 
of reducing the required time to 5 units. 

The reader will note that the use of indi- 
cator registers in thb way preserves the 
order of operations b followed by c when- 
ever con/Itcf (b, c). However, this use has the 
disadvantage that a wait for the satisfaction 
of- a condition by an* indicator will cause the 
issuance of instructions to halt temporarily, 
even though some successive instructions in 
the window might be issuable as operations. 
Hence, this method cannot be considered 
optimal. 

Note that such "wait conditions" may 
result because of register confiicta, or be- 
. cause of the unavailability of the function 
uiut. A scheme to partially nullify the latter 



tual Ft units. The scan passes to ih , even 
though at IB waiting. Although the time re- 
quired here is the same as in Figure 6, note 
that in Figure 8, Ft finishes earlier. This 
could be exploited, were there more instruc- 
tions to follow. 



THE EFFEa OF BUFFERING 

In many cases of practical interest, the 
actual function is computed from Imfert 
for the domain registera, rather than from 
the registers themselves. Indeed, rather than 
having the issuance of an operation transfer 
the indiui of the domain registerv to the 
function unit, the vafii«« in the dom^ regis- 
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FiooRc 8. Timing udns eimple indicator scheme 
with one f i unit ond l«n> virluol F% unito. 

* iera can be transferred. This atmplifieB the 
design* with a small increase in time for 
execution due to the added buffering. How- 
ever, it is likely that the function unit will be 
implemented with internal buffers, whether 
or not advantage is Uken of this fact, as 
discussed in the following paragraphs. 

The use of buffering has ramifications for 
increased concurrency. If 6 is the operation 

and this operation is really executed as 
i - fid, y) 

where z and y are buffer registers that are 
distinct from all program-addressable regis* 
ten, then we may relax the constraints on 
sequencing, provided the scan proceeds only 
after buffering has taken place. If c is an 
operation that follows b, then recall that we 
required 

D{b) n R{c) " 0 
Die) n R{b) - 0 
and R{b) n R{c) - 0 

for concurrent execution of b and e. How- 
ever, if we start c only after the buffering 
operations for b 

have been done, we no longer need the con- 
straint 

Dib) n R{e) « 0 

because operBtion e cannot possibly affect 
the computed value of b. If buffering is done 
uniformly for all operaUons, then we see that 
the scheme can be simplified to require only 
a one-bit indicator for each register, with 
that bit indicating whether an operation 
which has the corresponding register in its 




FiauRC 9. Tb« P and "covere" relotion for Ex- 
ample I with instantADCous domain and rangp 
buffering. 

range is in progress. This is essentially the 
scheme described in (Tol (pp. 2&-29). 

It is not difficult to show that buffering 
the outpul of a function unit can similarly 
remove the requirement Rib) (1 H(c) « 0. 
That is, if R{b) is buffered for use in the do- 
main of any operation iasued before c, then 
the range conflict requirement need not be 
of concern. Of course the requirement Rib) 
f1 D{e) «i 0 can never be xemoved, as this 
indicates a logical dependency between b 
and c. 

In summary, if we can be sure that buffer- 
ing occufB before the scan proceeds to the 
next instruction, then the P relation becomes 
Pisi , «/) if, and only if, i < j and Riti) H 
9^ 0. 

Figure 0 shows the eooero relation for Ex- 
ample I when both domain and range buffer- 
ing are used, assuming that buffering effee* 
tively occurs instantaneoualy. The conflict 
relation, as defined in the section on "Theo- 
retical Basis" is not meaningful in this case, 
since precedence now depends on the origi- 
nal ordering, as well as on which registers are 
used. We leave to the reader the task of 
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formulating a suitable analog of "conflict," 
aa well aa that of constructing the corre- 
sponding relation that explicitly shows 
buffering operations. (Note the presence of 
fewer arcs in Figure 9 than in Figure 5 
(page 000), indicating the greater degree of 
concurrency possible.)' 



fORWAROING 

To prevent the issuance of operations from 
being halted by register conflicts, an ap- 
proach called JonDarding can be used [To]. 
Our presentation is modifled from that in 
|To| for clarity. (It is unfortunate that the 
description of the scheme in the cited paper 
is Ubeled with the implementation detail of 
a "common data bus." Indeed a bus, or any 
of several other possible means of intercon- 
nection, could be used to route the data 
. between registers and fimction units. Our 
description ignores this detail.) 

In this scheme, a register may contain a 
data value aa before, or a specially-indicated 
"tag." A lo^ is simply an index of one of the 
virtual function units. If a register contains 
a tag, its proper contents have not yet been 
computed. The tag is, in fact, the index of 
the function unit from which the computed 
value will come. 

We assume that each virtual function unit 
has domain bufleis. Forwarding schemes 
without buffers will not differ much con- 
ceptually. With forwarding, if the OIU wishes 
to issue an operation b with t € D{b), but 
register t contains a tag, the unit eondu 
tiatially issues the operation and passes the 
tag to the virtual function unit speciBed in 
the operation (or to the function unit if 
virtual function units are not used). This tag 
is kept in a bufler of the virtual function 
unit as an indicaUon that the operation is 
not to proceed until the function unit cor- 
responding to the tag completes its exe- 
cution. Upon completion, the control checks 
the registers and buffers within all virtual 
function units, to see if any of their contents 
match the tag of the completing function 
unit. If a match occurs in the registers, the 
result of the completing operation is sent to 
the proper register. If a match occurs in the 
buflers, the result is foneardtd to the condi- 



tionally-issued operation. Whtin a condi- 
tionally-issued operation has all of the 
necessary operands, it is considered to be 
iteued and may begin execution. The execu- 
tion of part of Example I, using forwarding, 
is shown in Figure 10. 

The primary advantage of the forwarding 
technique is that the examination of instruc- 
tions does not atop simply bceauae an in- 
struction with a busy register is encountered. 
This means that if there are enough virtual 
function units, all potential concurrency 
will be detected without stopping the scan. 

The main disadvantage with this imple- 
mentation is that forwarding requires an 
associative search to match tags; this may 
either be time-consuming or require rather 
complex hardware implementations. The 
reader might observe that the need for associ- 
ative aearchea could be overcome if we had a 
way of associating with each virtual function 
unit a list of registers to which its results 
are to be sent. Such a list might be imple- 
mented using a linked-list strategy, for ex- 
ample. However, there are some subtieties 
that limit the usability of this approach. The 
fact that a range register containing a tag 
may be overridden (e.g., in Figure 10) indi- 
cates that there will be difiiculties in up- 
dating these lists. If the number of registers 
and buffers is small, it becomes feasible to 
lise a bit vector to represent the register to 
which the results should be forwarded. Up- 
dating these bit vectors is substantially 
simpler than updating a linked-list. Other 
organizations which eliminate the associa- 
tive search are discussed in the next section. 

It is clear that the scheme using forward- 
ing is optimal (provided there are enough 
function units) since an operation e is not 
executed if, and only if, there is an operation 
h which is either being executed or pending 
execution, such that eonJliel{b, e). That is, 
the uBuance of operations is never halted 
because of a register conflict. Furthermore, 
fonvarding incorporates domain and range 
buflfering quite naturally. Because the 
forwarding scheme ia combined wth buffer- 
ing, the indicators H',- are totally redundant, 
since If. « 0 if, and only if, Ri does not 
contain a tag. 

Figure 11 depicts the timing diagram for 
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Figure 10. Timing diagram for exeeulioit of pari of Example I using foirarding with three fi unite, 
itro F% uniU, and one £^ uniL 

Example I with -virtual function units and 
forwarding using only one real Fi unit. Fig- 
ure 12 dcpicta the same example using two 
real Fi units. As before, we assume for illus. 
tratton that each operation requires one 
time unit. The opttmaHty of this scheme is 
veri5ed in Figure 12, as the absolute mini- 
mum time (4 unita) ia achieved, as indicated 
by the longest path in the graph in Figure 5 
(page 181). 



i t I 1 



OPTIMAL SCHEMES WfTHOUT ASSOQATrVE 
SEARCH 

In the previous section we mentioned the 
difficulties of eliminating the associative 
search that accompanies the forwarding 
scheme. Here we discuss other organisations 
that eliminate the search, and which were de- 



I 

t-O 1 3 

FiouRR 11. Timing for Exunple I with for- 
vrarding and one Ft unit. 

velopcd in more theoretical contexts. On a 
practical-implementation basis, these tech- 
niques may not be competitive with those 
discussed in the previous section. However, 
they provide a useful conceptual tool, and 
will be seen to fit the need nicely when we 
introduce the consideration of deci«ons. 
We assume, for initial simplicity, that do- 
main buffering will not be used, then modify 
this assumption later on. 
It was shown in |Ke| that optimality 
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could be achieved by a control scheme that 
employs firat-in>5rBt-out queuu in the fol- 
lowing manner. One queue xb associated with 
each pair of conflicting operations. We will 
say that an operation belongs to a queue if 
it is one of the operations with which that 
queue is associated. 

We think of the elements stored in a queue 
as being tokeru, with a different token being 
associated with each operation. When the 
operation -issuing unit encounters an instruc- 
tion, it simply places one token for the cor- 
responding operation at the toil of each queue 
to which that operation belongs. Before an 
operation can begin, there must be a cor- 
responding token at the head of each queue 
to which it belongs. When the operation 
completes, the tokens are rennoved. Each 
queue can be implemented as a linked-list. 

It is easy to see that this scheme works be- 
cause it preserves the necessary precedence 
between conflicting operations. It Is optimal 
because it preserves only this precedence. 
An unfortunate property of this scheme is 
that the number of queues may be prohibi- 
tive, since, if there are m different binary 
functions and n different registers, we may 
have on the order of mn' different opera- 
tions, and (it can be shown) on the order of 
mn* conflicts pairs. 
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FiQURC 12. Timias for Exftmple I with for* 
vardiog aad two f| unitA. 



To reduce the number of queues from one 
queue per conflict pair, we m&y use one 
queue for each of a set C| , Ci , • • • , C* . 
where each C, is a set of operations, 
provided that for every pair of operations 
6, c. conjlieiib, c) if, and only if, there is an t 
such that|6, e| C C. . As before, if 6 € C« . 
then we say that 6 belongs to the correspond- 
ing queue, and the description of the control 
mechanism holds aa stated previously. This 
approach is illustrated in Figure 13. 

We note that such a set of queues can al- 
ways be obtained by associating one queue 
with each pair (6, i), where i is a register and 
6 is an operation such that i € D(b). The 
operations that belong to this queue are 6, 
together with those c, siich that t 6 R{c). 
This reduces the maximum number of 
queues to mn. 

We note that buffering may be used with 
the queueing scheme in a natural way. Each 
operation 6 is split into three parts, b| , b, , 
and b, , such that bi corresponds to buffering ' 
one domain register, bg corresponds to buffer- 
ing the other domain register, and b| cor- 
responds to storing the result The conflict 
relation can then be defined between parts of 
operations, rather than between the oper- 
tions themselves, and queues can be de- 
fined accordingly. 

A similar queueing scheme is discussed in 
(Del. Here there is one queue for each regis- 
ter. Any operation b for which i € Z>(b), or 
i € R{b) can appear as a token on the queue 
corresponding to t. However, the queue is 
not strictly first- in -first-out. Instead, if b 
and e are operations with t ^ D{b) U Z)(c), 
but i C R{b) U /2(c), then the tokens cor- 
responding to b and c on the queue corre- 
sponding to i can be interchanged arbitrarily. 
However, if i € D(b) U A(b), and t 6 A(c}, 
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Fiocftx U. Queueing scheme applied to Eiample 1. 
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then the tokens corresponding to 6 and c 
cannot be interchanged. 

The number of queues in this vereion may 
be subsUntiftlly fewer than in the previous 
scheme, because there is only one queue per 
rt^gister. However, the fact that token inter- 
changing can occur in a nondctcrminietic 
fashion casts doubt on the efficiency of such 
an implementation. Fortunately, this scheme 
can be rescued by using domain buffering 
and virtual function units, as described 
earlier. A modification of this type is dis- 
cussed next. 

Whenever an operation b token appears 
at the head of the queue for register i, with 
« € /)(6), a virtual operation is set up, im- 
mediately transferring the contents of 
register i into the domain buffer for this 
operation. We know that the contents of 
register t are valid, since i appeared at the 
head of the queue. The token is then re- 
moved from the queue, and similar buffer- 
ing can occur for other operations. When its 
domain buffers are filled, a virtual opera- 
tion can begin. If a range token is at the 
head of the queue, the operation can be 
issued; but the token is not removed, and 
further tokens cannot be examined until 
this operation has completed. The advantage 
of this modification is that no interchanging 
of tokens is necessary. Unfortunately, to be 
completely competitive with forwarding, the 
ability to do range buffering must be rein- 
troduced. This has the effect of again multi- 



plying the number of queues, since there 
would be one queue for each range buffer. 
Figure H Uluatrates the second queueing 
techmque. 

Aside from implementation details, the 
difference between the schemes discuaslxl in 
this section and the forwarding schemes 
diacuased in the previous section is mainly 
one of viewpointa. On the one hand, we view 
the control of eequendngaa being distributed 
among the virtual function unite, and on the 
other hand as being present in a "global" 
control unit 



THE CASE FOR DEOS/ONS 

Let us now consider what happens if de- 
cisions are allowed in the window. Obviously 
this means that leas instruction-fetching 
has to be done in the case where a decision 
causes looping back to another instruction in 
the window. This is one very practical reason 
for including decisions in the window. 

We now consider what it means to perform 
"look-ahead" when decisions are involved. If 
c is a decision, then it is clear that the cxecu- 
tion of e must be deferred if there are any 
operations 6 with R{b) 0 D{e) 9^ 0 which are 
either pending or in execution. 

Since a decision affects the flow of control 
in sequential execution, look-ahead past a 
decision is normally limited. One possibility 
is that look-ahead could proceed through 
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FiooRa 14. The oeeond queuelns aeheme appllsd to Example I. 
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both alternatives of a decision in parallel » 
and when the decision is finally complete, 
the results of the operations m the proper 
alternative would be kept, and the other 
results destroyed. The control in this case 
would be extremely complicated, and extra 
function uiiits would be required to do the 
"parallel" look-ahead at the same speed as 
the "serial" look>ahead. Also, if any alterna- 
tive itself contains a decision, the problem 
growB rapidly out of proportion. We assume 
that this type of look-ahead is not used, even 
though wc acknowledge the fact that it may 
result in some speed-up. We make a similar 
aaaumption for any scheme which "guesses" 
one of the alternatives and conditionally 
executes the corresponding operations. We 
claim the additional hardware costs that 
would be incurred in all these cases are not 
justifiable. 

Having made these assumptions, what is 
left to be considered? First, it is possible 
that some operation will be executed regard- 
less of which particular alternative of a de- 
cision occurs. Furthermore, this operation 
may be such that its operands are available 
before the decision is executed. Hence, such 
an operation may be "pulled," or "per- 
colated" through the decision and executed 
before, or concurrently with, the decision. 
We show in [Ke] that such operations, if 
they are not decisions, can be detected and 
percolated by preprocessing the program. 
This is illustrated in Figure 15. If percolation 
is done prior to execution, then the task need 
not be performed by the operation-issuing 
unit. In fact, we see no practical way of 
handling this other than by preprocessing. 

If the operation to be percolated is a de- 
cision, the problem is greater because of the 
possibility of interchanging, or concurrently 
executing, two or more decisions. Matters 
then become complicated because we have a 
total number of aJtematives that is the pro- 
duct of all the individual alternatives. We 
have not yet found a way to handle all of 
these conveniently by a look-ahead mecha- 
nism. Hence, we make the following assump- 
tion. 

Atoumplion 4: No two decisions are exe- 
cuted concurrently, and each is executed in 
the order specified in the original program. 





c=0 




FiGUAK 16. UlufltratinB "pereolaUoa"— no op- 
eratioaa are conflietiog. 

Techniques that account for concurrent 
execution of decisions have been described 
[Da], but the feature that excludes those 
techniques from practical consideration here 
is that parallelism is explicitly specified to a 
nukchine capable of interpreting the speci- 
fication, rather than it being implicitly 
specified, as in the case of a look-ahead 
processor. 

A subtle point that occurs when decisions 
are permitted, even with the restrictions 
stated above, is that previously-issued opera- 
tions can be executing or pending while a 
decision is executing. This means that the 
execution of operations from two different 
iterations of a loop may overlap in time. We 
are then led to the observation that, con- 
trary to the case in which there are no de- 
cisions, no finiU control can be opiimal when 
decisions are permitted. A formal proof of 
this fact is given in [Ke], so here we present 
an informal and more intuitive version. Ob- 
serve the following program : 

«»: l-f.(l.l) 

«a : . If G,{2) then go to «i 



Although this program ,may seem trivial, or 
somewhat contrived, it abstracts a situation 
that may occur in more complex examples. 
Presumably, registers 1 and 2 have been 
suitably initialixed. Suppose the execution 
time of the operation corresponding to oi will 
always take nominally longer than 0| and 
di combined. An optinaial look-ahead control 
will note that whenever a» is executed and 
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FiouRR 16. niuatratlns the execution of a pro- 
HTum in vrhieh a. larc« number of pending exe* 
cutions of an operation need to be remembered 
by ihe eonirol. 

the condition C«(2) la satisfied, resulting in a 
transfer to Oi , then Oi and the sequence oi 
followed by si can be processed concur- 
rently. Suppose Si is executed with the out- 
come being a transfer to ai , but the previous 
operation generated by 0\ has not yet com- 
pleted. Since S| uses register 1, the next op- 
rjration to be generated by ai cannot begin, 
although fli can begin. For maximal paral- 
lelism, ffi must be issued, and therefore the 
control must remember the pending execu- 
tion of <! . Now if the first S| is sufficiently 
slow, the second Si . and then si will both be 
completely executed before the first Si com- 
pletes. If the test G,{2) is again satisfied, 
then control must remember the pending 
execution of two copies of ti , and so on. Even 
if some copies of do completely execute as 
time goes on, the control may have to re- 
member that arbitrarily many copies of 
are pending. But no finite control can count 
to an arbitrarily high number, therefore no 
finite control is optimal. Figure 16 shows 
part of a timing diagram in which the pend- 
ing execution of a large number of «i have 
to be remembered by the control. 

By using an argument similar to that in 
the preceding paragraph, it can be demon- 
strated that no control which uses only a 
fixed number of counters, even if these are 
allowed to be unbounded, can be optimal 
[KeJ. However, it is shown in the same 
reference that if queues of unbounded length 
are allowed, then under the restriction on de- 
cisions stated earlier, an optimal look-ahead 
can be constructed. 

The fact that an optimal control can be 
constructed with unbounded queues may not 
appear to be of much consolation to the 
system designer. However, the construction 
technique docs offer valuable conceptual 



information on the organisation of a finite 
control. 

Our discussion shows that when using 
look-ahead schemes, the lengths of queues 
are arlifieiaUy bounded by implementation 
considerations. That is, for some programs 
it is always possible to get a greater speedup 
by adding more control states. Tradeoffs in- 
volving lengths can be determined by simu- 
lation of typical instrucUon streams. 



SCHEDUUNG 

Thus far we have been concerned only with 
the detection of parallelism as it occurs in 
iasuino operations. I f each issued operation 
can be immediaCeiy^ assigned to a real func- 
tion unit, we would then have the maximal 
degree of parallelism allowable by the par- 
allelism-detection meehanism. 

The insurance that there are enough func* 
tion units avulable for optimality would 
probably result in idle units a large percent- 
age of the run-time, as it is unlikely that alt 
types of functions will be used with constant 
frequency. However, if there are fewer real 
function units than generated operations, 
then some choices must be made concerning 
the order of execution of the operations. This 
problem of tehedvling operations has an 
arbitrary solution insofar as logical depend- 
encies among instructions are concerned, but 
it can have an effect on the time required to 
execute a program. This is because the order 
of execution of operations may determine 
the order in which subsequent operations 
can be issued, and thereby determine whether 
certain function units will be idle. Some subt- 
leties of scheduling are discussed in |G1, 02). 

One solution to the scheduling problem is 
Ui have the compiler order its code so an 
efficient execution is obtained when the code 
is executed, according to scheduling on a 
fixed basis, say first-in-first-out. In other 
words, the first virtual unit in a specific 
ordering whose operands are available Is the 
next to be assigned to a real function unit. 
Although such ordering by a compiler might 
be possible, there are a number of draw- 
backs: 

1) such a procedure necessarily assumes 
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a sUUe window, and therefore may be 
of questionable validity; 
2} very few cases exist for which fast 
scheduling algorithms are known. 
Furthermore, recent work (U) has in- 
dicated that scheduling algorithms 
are generally prohibitively time-con- 
suming; and 
3) the code produced is likely to be 
highly machine-dependent. Tliia may 
-be undesirable. 
Thus, it appears that preprocessing by the 
compiler is best limited to fast heuristies 
that are very likely to increase concurrency. 
Some bounds on the worst case for such 
heuristics in the instance of identical func- 
tion units are derived in {Gl]; these appear 
encouraging because they indicate that 
trivial scheduling schemes can produce exe- 
cutions that differ from the optimal by at 
most a small multiplicative factor. However, 
similar results have not been shown for non* 
identical function units. Derivation of 
bounds in this more general case is a problem 
for future research. 

We may ask similar questions about dy- 
namic scheduling wilhin the window itself. 
Certainly if fast compiler techniques for 
scheduling are difficult to 5nd, then the 
same is true for fast and sufficiently inex- 
pensive dynamically-executable scheduling 
techniques. However, there b a subtle dis- 
tinction. Scheduling within a machine may 
be made to depend on the precise knowledge 
of which resources are available at any in- 
stant of time, and the stratefQr may be 
varied accordingly. The effectiveness of 
dynamic scheduling remains open for future 
investigation. 



ALGEfiRAJC IDENTmES 

This and the following section survey other 
possible enhancements to look-ahead mecha- 
nisms. We make no attempts to present 
optimal methods when these enhancements 
are allowed, as the problem of doing so ap- 
pears intractible. 

According to Assumption 2, (page 180} we 
have been assuming that no nontriviat re- 
lations hold between different operations. 



Now we consider the relaxation of this con- 
straint. 

Tint, let us consider the case in which two 
or more functions with different names are 
in fact equivalent. This may be useful in 
allowing the programmer to bypass any 
built-in scheduling mechanism, enabling him 
to preplan the schedule for greater parallel- 
ism. For example, if thej-e are three adder 
units, then the programmer may use a 
different function code for adder 1, adder 2, 
or adder 3. The specification of a particular 
adder would indicate that if the adder is 
busy, the operation is not to be issued, even 
if another adder is available. It is not diffi- 
cult to provide examples wherein this form 
of "balking** results in reduced execution 
time. A special function code might be used 
to indicate that the programmer doesn't 
care which unit is used, and the choice can 
be made arbitrarily and dynamically. 

Another type of relation among operations 
ia associativity and/or commut&tivity of, 
for example, addition or multiplication. Al- 
though associativity docs not hold for either 
of these operations in the domain of floating- 
point numbers, it xn&y be assumed, with the 
knowledge that the results of a series of such 
operations may not be precisely determined. 
It has been observed, e.g., in |ICMC), that 
associativity and commutativity allow possi- 
ble speedups in the execution of programs 
for arithmetic expressions without intro- 
ducing any additional operations. On the 
other hand, the assumption that multipli- 
cation distributes over addition can be used 
to effect a speedup, but additional opera- 
tions must generally be introduced. There- 
fore, in a case where the number of real func- 
tion units is a limiting factor, this speedup 
may not materialize. This indicates that a 
rather machine -dependent compiler may be 
necessary to take full advantage of the avail- 
able resources. Alternatively, it may be 
possible to have the control dynamically 
decide whether to apply an algebraic identity, 
depending on the availability of real func- 
tion units. To the author's knowledge, no 
efficient techniques for accomplishing this 
are known. 

With regard to the type of dynamic de- 
cisions mentioned here, however, one tech- 
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niquc we would like to suggest ta the use of 
more complex inBtnicliona. For exAmple, a 
single instruction might specify "add the 
contents of the next 5 registers, listed.'* If 
there ia no implied ordering of the operpnds, 
then associativity and commutativity are 
being assumed. The contents of some regis- 
tera might not yet have been compuUd, but 
a Bet of adder units could go to work on 
those that have, adding pairs of operands as 
they become available. This gives a more 
flexible ordering than is possible with the 
technique of requiring a predefined sequence 
of additions. 



OTHER CONSIDERATJONS 

We now mention some other factors that 
relate to the effectiveness of a look-ahead 
processor. First, there is a general iiation of 
forwarding and buffering to allow arbitrary 
reffiiUr renamiTig. 

It may be observed that there is nothing 
particularly sacred about the register in 
which a value is stored. The index of a 
register is simply a name by which that value 
can be accessed when fetching is necessary. 
Thus, the physical registers used are really 
arbitrary, as long as we have a way of re- 
calling the value as30ciat<*d with a particular 
name. This indicates that it is possible to 
reduce register conflicts by dynamically re- 
naming registers. 

For example, suppofle an instruction spe- 
cifies that a cerUin value is stored in register 
i. Suppose, also, that some instruction which 
appean several instructions later in the 
stream also specifies storage into t. The latter 
instruction cannot normally be executed in 
advance because instructions between it and 
the former instruction may reference register 
t. However, the following scheme may be 
used to allow intermediate instructions to 
proceed in parallel with subsequent instruc- 
tions, provided that no further conflicts arise. 

Suppose we associate a unique index vriiAi 
each physical register, and consider the in- 
dices specified in instructions as namM for 
these rcgiatere. In general, there are to be 
more registers than names. In addition, we 
assume that there is a mappiiiQ table which 
gives for each name the index of the physical 



register with which the name is currently 
associated. When an operation involving 
register t is issued, t is translated into the 
index of the physical register currently as- 
signed to u Henceforth, the operation ad- 
dresses this register through its physical 
index. 

When register i is specified as the name of 
a range register, a new physical register b 
assigned, and the mapping table is updat-ed 
to reflect this. The fonner physical register 
is now inaccessible to future instructions. So 
the control kno%ra when an inaecesable 
physical register is to be reasmgned, a count 
similar to Wt in the section on "Elementary 
Schemes" must be associated with each 
physical register, indicaUng how many 
operations have been issued that will refer- 
ence its value. As each reference occurs, the 
count is decremented, and when it reaches 
zero, the register is available for reassign- . 
ment. Observe that with a general renaming 
scheme, it is unnecessary to have more than 
one register name in order to make use of 
multiple registers. Thus, even a single- 
accumulator architecture will suffice. Stone 
[8, E] has Uken thia a step further by sug- 
gesting the use of a renaming scheme in con- 
junction with an ^'addrcssless'! pushdown 
stack machine. With increased use of cache 
memory organizations, which remap regis- 
ters in any case, the renaming scheme be- 
comes increasingly attractive. 

The technique of muUipU tMiruetion 
tireavu may be used to increase the effi- 
ciency of a look^ahead processor. Since it is 
not likely that a typical instruction stream 
can make use of all of a large collection of 
function units and/^or registers, we may al- 
low several different operation-issuing units 
to issue operations from different instruction 
streams to a pool of function units. Each 
stream is associated with a separate instruc- 
tion pointer and decoding unit. This has 
been suggested in (AFR, FPSl, for example. 
MulUstream organisations without inter- 
active sharing are presented in [H, Thl. 

The principal usefulness of the multi- 
stream technique is based on the assumption 
that several streams are likely to make more 
uniform use of the pool of function units 
than one stream. As with several other tech* 
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niques' which we have discussed, the in- 
creased complexity of the control may out- 
weigh the gain in efficiency. This is due in 
part to the requirement that there be mecha- 
nisms for resolving conflicting requests for 
function units. Also, when accompanied by a 
register renaming scheme, "deadlocks" can 
result, as mentioned in [Co]. 

Another concept which relates to those 
wc have diacuased is that of "pipelining" 
(HT, Wat). By this term we mean the execu- 
Uon of a sequence of umilar operations in a 
way that allows conciurent computation of 
different suboperations of more than one 
operation. For example, a floating-point 
addition typically consists of three phases: 
alignment, fraction addition, and normaliza-- 
tion. Suppose each phase is considered a 
distinct suboperation. performed by a dis- 
tinct subfunction unit. Then at most one 
subunit will be used at a time for processing 
any given operation, and hence this unit can 
be used to process suboperations of other 
operations. We can consider the three sub- 
function units as forming a "pipe" through 
which operations flow. Aasuming each sub- 
operatton requires the same amount of time, 
if there are n distinct suboperations, then 
the net time to execute a large number of 
operations is roughly 1/n of the time re- 
quired for sequential execution. 

Pipelining relates to our discussion of 
look -ahead in two waya. The Brat is that 
look-ahead itself may be conudered a form of 
pipelining in which the operations can be 
highly-diaaimilar, provided operations are 
scheduled on virtusJ function units on a 
first-in-first-out basis. Second, a function 
unit may itself be implemented as a pipeline 
that is fed by the corresponding virtual 
function units. 

An interesting coupling of pipeline and 
look-ahead processing concepts is discussed 
in ICr). Here the regiaters are capable of 
holding vectors of operands. Look-ahead 
control can be organized as we have dis- 
cussed earlier. However,, by appropriate 
oyochronization, buffering can be achieved 
with single component buffers, instead of by 
the obvious approach of buffering entire 
vectors. 

The techniques of register renaming, pipe- 



lining, and multiple-streams have prompted 
some authors to consider more radical 
machine organizations [DM, MC, MT, Roj. 
This has led to the data jUno programming 
concept, in which a program ia specified as a 
graph, similar to the precedence graph, 
rather than as a sequence of instructions. 
' The idea is to eliminate the "intermediate" 
sequential program from the machine-inter- 
pretation phase of problem solution. The 
concept apparently originated theoretically 
in|KMl). 

As our final conaideration, we should men- 
tion that any potential increase in perform- 
ance can be shattered if the instruction 
stream ia subject to frequent interrupts. The 
reason for this, of course, is that when an 
interrupt occurs, if the interrupt routine is 
to be able to use programmable registers, 
then all operations in progress must be com- 
plete before the regiater contents can be 
saved and the instructions in the interrupt 
routine can be processed. This grows more 
complicated if the interrupts are due to some 
aspect of the execution of the operations 
themselves, such as the occurrence of an 
arithmetic overflow. The latter considera- 
tion has led to the notion of "imprecise in- 
terrupt" |AST|. This means that intcrrupU 
which cannot be precisely associated with 
any one instruction are allowed to occur, but 
the general vicinity of the instruction is 
known. In the machine described in (AST), 
for example, this feature can be turned off 
and instructions processed serially. 

The interrupt problem can be alleviated 
in part by using a flexible register renaming 
scheme, such as that described earlier. How- 
ever, it is probably a better idea to decrease 
the frequency of interrupts, handling them 
on a separate "peripheral" proceasor if 
possible. For a comparison of approaches, 
see (AST, Th). 
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